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DETAILED ACTION 

Response to Amendments and Arguments 

1 . Applicant's response to the last Office Action, filed Jan 28 th , 2005, has 
been entered and made of record. 

2. Applicant has not amended any claims. Applicant has added new claim 

21. 

3. Applicant's arguments have been fully considered but are not persuasive 
for at least the following reasons. 

4. With regard to claim 1 , Applicant argues that the reference of Nakagawa 
does not disclose the image interpolation as claimed in claim 1 . Applicant submits that 
Nakagawa performs interpolation between images using a different approach than 
Applicant's claimed invention. Nakagawa performs interpolation, using images of a 3D 
object from different viewpoints and interpolating between the viewpoints using angles 
and a trajectory of points. Applicant's present invention interpolates an image by 
acquiring two image pairs on two different axis and interpolating to generate an 
intermediate frame or image. Applicant also argues that Nakagawa does not disclose 
or teach the feature of "acquiring a second image pair, comprising two key frames, and 
second corresponding data point between the two key frames of the second image pair" 
as claimed. Examiner points to the discussion of the previous rejection in regard to this 
step in claim 1 . Nakagawa discloses a method wherein known points of view of a three 
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dimensional object are interpolated between to generate interpolated images of the 
object (column 3, line 65-column 4, line 16).- It is therefore understood that a first pair of 
images are interpolated between to generate a first interpolated image or data for an 
interpolated image then another pair are interpolated between to generate another 
interpolated image or data for an interpolated image and then the two new interpolated 
images are interpolated between. 

Applicant also argues that Nakagawa does not disclose the feature claimed in 
claim 7 in which "the key frames used for interpolating are photographed respectively 
from a same view point but at different times." Examiner submits that the motion of 3D 
object is determined according to multiple frames of the image (column 4, lines 35-50). 
Here it is understood that the key frames involve the object in motion, which is 
interpreted to involve images at different moments in time. Examiner submits that the 
reference of Nakagawa broadly interpreted still reads on the present invention as 
claimed. Therefore the rejection is maintained and is accordingly made final. 



Claim Rejections - 35 USC §112 

5. Claim 21 is rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. In the Applicant's remarks on page 13, 
Applicant states that the references do not teach or suggest the advance calculation of 
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corresponding point files as described at page 65, lines 20 to 26 in the specification and 
claimed in claim 21 . However Examiner submits that the advance calculation of point 
files is not taught on page 65, lines 20-26 or anywhere else in the specification. Also it 
is unclear how it is even possible to have first corresponding point data and second 
corresponding point data calculated in advance of acquiring or at least knowing the 
respective image pairs. Clarification is required as to what the calculation of first and 
second corresponding point data is in advance of and how it is possible to calculate 
point data between image pairs in advance of acquiring the image pairs. 



Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this Office 
action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

7. Claims 1-11 and 13-20 are rejected under 35 U.S.C. 102(b) as being 
anticipated by U.S. Patent 5,831 ,619 to Nakagawa et al. 

8. With regard to claim 1, Nakagawa discloses an image interpolation 
method, comprising acquiring a first image pair, comprising two key frames and first 
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corresponding point data between the two key frames of the first image pair (column 2, 
line 61 -column 3, line 22). Nakagawa discloses how multiple images are taken of an 
object from multiple viewpoints and those viewpoints are then interpolated to create an 
image of the object from a specified viewpoint. 

Nakagawa further discloses acquiring a second image pair, comprising two key 
frames, and second corresponding point data between two frames of the second image 
pair (column 2, line 61 -column 3, line 22). It is understood that Nakagawa discloses 
determining an image of the object from any possible viewpoint by interpolating 
between known viewpoints. The multiple known viewpoints are interpreted as being 
grouped into pairs that are then interpolated between. 

Nakagawa further discloses generating an intermediate frame by interpolation, 
wherein the interpolation utilizes positional relations of a first axis and a second axis and 
the first corresponding point data and the second corresponding point data (column 3, 
line 65-column 4, line 16). Nakagawa discloses a method in which known points of view 
of a three-dimensional object are known and other intermediate images of the object are 
determined by interpolation. It is interpreted that Nakagawa would determine the 
viewpoint in between four different viewpoints or two axes by interpolating between first 
one pair of images and then the other pair and then interpolating between the two 
interpolated images. 

Nakagawa further discloses the first axis being determined temporally or spatially 
between the second axis being determined temporally or spatially between the two key 
frames of the second image pair (column 4, lines 16-40). Nakagawa discloses 
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interpolating between spatial viewpoints and also discloses interpolating between views 
of an object in motion or a temporal interpolation (column 4, lines 35-50). 

The second axis being determined temporally or spatially between the two key 
frames key frames of the second image pair (column 4, lines 16-40). Nakagawa 
discloses interpolating between spatial viewpoints and also discloses interpolating 
between views of an object in motion or a temporal interpolation (column 4, lines 35-50). 

9. With regard to claim 2, Nakagawa discloses a method wherein the first 
image pair and the second image pair are determined so that the first axis and the 
second axis do not lie on the same line (column 3, lines 40-45). Nakagawa discloses 
interpolating between known points on curves or ellipses around a 3-D object and the 
different curves are interpreted as different axes. 

10. With regard to claim 3, Nakagawa discloses a method wherein one of the 
two key frames in the first image pair and one of the two key frames in the second 
image pair are common, and the interpolation utilizes positional relations based on a 
triangle having the first axis and the second axis as two sides thereof (column 3, lines 
40-45). Nakagawa discloses interpolating between known points on curves or ellipses 
around a 3-D object and the different curves are interpreted as different axes. It is 
understood that the interpolation axes can fall anywhere on the sphere and can 
intersect forming a triangle for interpolating. 
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1 1 . With regard to claim 4, Nakagawa discloses a method wherein the first 
image pair and any of the two key frames in the second image pair do not have any key 
frames in use, and the interpolation utilizes positional relations based on a quadrilateral 
having the first axis and the second axis as two sides opposite to each other (column 3, 
line 65-column 4, line 16). Nakagawa discloses a method in which known points of view 
of a three-dimensional object are known and other intermediate images of the object are 
determined by interpolation. It is interpreted that Nakagawa would determine the 
viewpoint in between four different viewpoints or two axes by interpolating between first 
one pair of images and then the other pair and then interpolating between the two 
interpolated images. 

12. With regard to claim 5, Nakagawa discloses a method and apparatus that 
interpolates between object views or images according to already known viewpoints or 
images (column 2, line 61 -column 3, line 22). Therefore it is understood that the 
apparatus of Nakagawa is capable of interpolating according to any known images and 
at any ratio between known viewpoints and the corresponding axes as desired to find 
any image for a viewpoint (column 8, lines 53-58). 

13. With regard to claim 6, Nakagawa discloses a method further comprising 
acquiring a positional relation between the intermediate frame, the two key frames of 
the first image pair and the two key frames of the second image pair, wherein the 
interpolation is performed based on said positional relation (column 3, line 65-column 4, 
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line 16). Nakagawa discloses a method in which known points of view of a three- 
dimensional object are known and other intermediate images of the object are 
determined by interpolation. In performing interpolation Nakagawa bases the operation 
on positional relation between points (column 3, lines 5-22). 

14. With regard to claim 7, Nakagawa discloses a method wherein the two key 
frames of the first image pair and the two key frames of the second image pair are 
images photographed respectively from a same view point but at different times (column 
4, lines 35-50). Nakagawa discloses how the motion of a 3D object is determined 
according to multiple frames of the image. 

1 5. With regard to claim 8, Nakagawa discloses a method wherein the two key 
frames of the first image pair and the two key frames of the second image pair are 
images photographed respectively from different viewpoints (column 3, line 65-column 
4, line 16). 

16. With regard to claim 9, Nakagawa discloses an image interpolating 
method, comprising computing a matching between a first image pair comprised of two 
key frames, and detecting first corresponding point data between the two key frames of 
the first image pair (column 2, line 61 -column 3, line 22). Nakagawa discloses how 
multiple images are taken of an object from multiple viewpoints and those viewpoints 
are then interpolated to create an image of the object from a specified viewpoint. 
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Nakagawa further discloses computing a matching between second image pair 
comprised key frames, and detecting second corresponding point data between the two 
key frames of second image pair (column 2, line 61 -column 3, line 22). It is understood 
that Nakagawa discloses determining an image of the object from any possible 
viewpoint by interpolating between known viewpoints. The multiple known viewpoints 
are interpreted as being grouped into pairs that are then interpolated between. 

Nakagawa further discloses generating an intermediate frame by interpolation, 
utilizing positional relations of a first axis and a second axis, the first corresponding 
point data and the second corresponding point data (column 3, line 65-column 4, line 
16). Nakagawa discloses a method in which known points of view of a three- 
dimensional object are known and other intermediate images of the object are 
determined by interpolation. It is interpreted that Nakagawa would determine the 
viewpoint in between four different viewpoints or two axes by interpolating between first 
one pair of images and then the other pair and then interpolating between the two 
interpolated images. 

Nakagawa further discloses wherein the first axis determined temporally or 
spatially between the two key frames of the first image pair (column 4, lines 16-40). 
Nakagawa discloses interpolating between spatial viewpoints and also discloses 
interpolating between views of an object in motion or a temporal interpolation (column 4, 
lines 35-50). 

Nakagawa further discloses that the second axis is determined temporally or 
spatially between two key frames of the first image pair (column 4, lines 16-40). 
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Nakagawa discloses interpolating between spatial viewpoints and also discloses 
interpolating between views of an object in motion or a temporal interpolation (column 4, 
lines 35-50). 

17. With regard to claim 10, Nakagawa discloses a method wherein said 
matching is computed pixel by pixel between the two key frames (column 5, lines 45- 
50). Nakagawa discloses that interpolation is performed on the images of the 3D object 
using identical points on the object and the images stored in memory. This is 
interpreted as performing matching on a pixel basis. 

1 8. With regard to claim 1 1 , Nakagawa discloses wherein said matching is 
computed pixel by pixel based on correspondence between critical points detected 
through respective two-dimensional searches of the two key frames (column 5, lines 45- 
50). Nakagawa discloses that interpolation is performed on the images of the 3D object 
using identical points on the object and the images stored in memory. This is 
interpreted as performing matching on a pixel basis and the identical points are the 
critical points in each of the key frames. 

19. With regard to claim 13, the discussion of claim applies. Nakagawa 
discloses an image interpolation apparatus for use with the method (column 1 , line 63- 
column 2, line 17). 
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20. With regard to claim 14, Nakagawa discloses a method and apparatus 
that interpolates between object views or images according to already known viewpoints 
or images (column 2, line 61 -column 3, line 22). Therefore it is understood that the 
apparatus of Nakagawa is capable of interpolating according to any known images and 
at any ratio between known viewpoints and the corresponding axes as desired to find 
any image for a viewpoint (column 8, lines 53-58). 

21 . With regard to claim 1 5, Nakagawa discloses an apparatus further 
comprising a matching processor, which generates the corresponding point data 
(column 5, lines 45-55). The identical points disclosed by Nakagawa are considered 
matched and they are considered matched by a processor. 

22. With regard to claim 1 6, Nakagawa discloses an apparatus wherein the 
two key frames of the first image pair and the two key images of the second image pair 
are images photographed respectively from a same viewpoint but at different times 
(column 4, lines 35-50). Nakagawa discloses how the motion of a 3D object is 
determined according to multiple frames of the image. 

23. With regard to claim 17, Nakagawa discloses an apparatus wherein the 
two key frames of the first image pair and the two key frames of the second image pair 
are images photographed respectively from different viewpoints (column 3, line 65- 
column 4, line 16). 
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24. With regard to claim 18, Nakagawa discloses an apparatus further 
comprising a user interface by which to input externally a specification regarding a 
temporal or spatial position of the intermediate frame to be generated (column 2, lines 
1-17). Nakagawa discloses a specified desired viewpoint and it is understood that the 
desired viewpoint must be specified by some external means either programmed or 
manually. 

25. With regard to claim 19, the discussions of claim 1 and 18 apply. 

0 

26. With regard to claim 20, the discussions of claim 1 and 13 apply. 
Nakagawa's apparatus and method are considered executed with a computer program 
(Fig.1). 

27. With regard to claim 21 , Nakagawa discloses a method according to claim 
1 , wherein said first corresponding point data and said second point data have been 
calculated in advance of said acquiring ( 

Claim Rejections • 35 USC § 103 

28. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a 
person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 



29. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of U.S. Patent 5,831 ,619 to Nakagawa and U.S. Patent 6,018,592 to 
Shinagawa et al. 

30. With regard to claim 12, Nakagawa discloses a method according to claim 
1 1 and also discloses performing image matching, but does not disclose 
multiresolutionizing the key frames. Shinagawa discloses multiresolutionizing the two 
key frames by respectively extracting the critical points (column 3, lines 35-54). 

Shinagawa further discloses performing a pixel-by-pixel matching computation on 
the two key frames, at same resolution levels (column 3, lines 48-54). 

Shinagawa further discloses acquiring a pixel-by-pixel correspondence relation at 
a finest level of resolution while inheriting a result of a pixel-by-pixel matching 
computation in a different resolution level (column 3, lines 55-65). 

Shinagawa teaches that this method overcomes matching problems such as 
blurring of the image at lowered resolutions. Therefore the critical point is determined 
first and then the resolution is lowered in order to maintain matching of critical structures 
in the image (column 3, lines 1-34). Therefore it would have been obvious to one of 
ordinary skill in the art at the time of invention to use the multiresolutionizing of 
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Shinagawa for use in the image matching of Nakagawa in order to overcome problems 
with image matching such as blurring in low resolution images. 



Prior Art 

Other prior art considered relevant, but not relied upon is as follows: 

U.S. Patent 5,613,048 to Chen et al. is a method for determining images of a 
three dimensional object in between known images of the object. 

U.S. Patent 6,222,551 to Schneider et al. is a method for creating a three 
dimensional object view manipulation through use of projections of the object. 

U.S. Patent 6,525,731 to Suits et al. is a system for providing 3D views of an 
object for manipulation. 

Conclusion 

Applicant's amendment necessitated the new grounds of rejection presented in 
the Office Action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Wes Tucker whose telephone number is 703-305-6700. 
The examiner can normally be reached on 9AM-5PM. 

If attempts to reach the examiner by telephone are unsuccessful/the examiner's 
supervisor, Amelia Au can be reached on (703)308-6604. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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